Simultaneous Modelling of Covariance Matrices: GLM, Bayesian and Nonparametric Perspectives

We provide a brief survey of the progress made in modelling covariance matrices from the perspective of generalized linear models (GLM) and the use of link functions (factorizations) that may lead to statistically meaningful and unconstrained reparameterization. We highlight the advantage of the Cholesky decomposition in dealing with the normal likelihood maximization and compare the findings with those obtained using the classical spectral (eigenvalue) and variance-correlation decompositions. These methods, which amount to decomposing covariance matrices into “dependence” and “variance” components, and then modelling them virtually separately using regression techniques, try to emulate the GLM principles to varying degrees, though the latter two are not satisfactory since their “dependence” components are severely constrained. Examples such as Flury’s (1984, 1988) common principal components (CPC), Manly and Rayner’s (1987) common correlations in multi-group situations, Bollerslev’s (1990) constant-correlation, Engle’s (2002) dynamic correlation and Vrontos et al.’s (2003) full-factor multivariate GARCH models are used to motivate various computational issues, when the number of parameters grows quadratically in the dimension of the response vector. Once a bona fide GLM framework for modelling covariance matrices is formulated (Chiu et al., 1996; Pourahmadi, 2000), its Bayesian (Daniels and Pourahmadi, 2002), nonparametric (Wu and Pourahmadi, 2003), generalized additive and other extensions can be derived in complete analogy with the respective extension of GLM for the mean vector (Nelder, 1998).